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^ ' Abstract. Phase transitions between the quantum spin Hall and the insulator phases 

in three dimensions are studied. We find that in inversion-asymmetric systems there 
appears a gapless phase between the quantum spin Hall and insulator phases in three 
dimensions, which is in contrast with the two-dimensional case. Existence of this 
gapless phase stems from a topological nature of gapless points (diabolical points) in 
three dimensions, but not in two dimensions. 
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1. Introduction 

In the intrinsic spin Hall effect (SHE) [HE], an external electric field applied to a doped 
semiconductor induces a transverse spin current. It has been attracting special interest 
from the following aspects. First, it can produce spin current without breaking the 
time-reversal symmetry (T-symmetry), namely without magnetism or magnetic field, 
which may be potentially important for spintronics device applications. Second, the 
mechanism itself is dissipationless and thus it may open a way for spintronics devices 
with less power consumption. Third, the SHE is driven by the spin-orbit couping, 
which can be even larger than room temperature. Thus it is expected to survive even 
at room temperature, as has been confirmed experimentally [21 HJ [5]. In particular, 
recent experiments on platinum shows the largest spin Hall conductivity so far [5], of 
about 240f2~ 1 cm -1 at room temperature. This might be attributed as an intrinsic SHE 
caused by near-degeneracy near the Fermi energy [6]. 

Henceforth we restrict ourselves on time-reversal-symmetric systems. In relation 
to the SHE in conducting systems, there have been a growing interest in the SHE in 
insulating systems. The first proposal is spin Hall insulators [7]. It was shown that in 
insulators such as HgTe under uniaxial pressure or in PbTe the spin Hall conductivity 
is nonzero. In a sense, they are ordinary insulators with spin-orbit coupling. Another 
proposal is the quantum spin Hall (QSH) systems, both in 2D PEHH)] and in 3D [TT1[T2] . 
They are insulators in the bulk while the boundaries (i.e. edges in 2D or surfaces in 3D) 
are gapless and carry spin currents. They can be regarded as topological insulators; these 
gapless boundary states are topologically protected against T-symmetric perturbations 
[T3| [H]. Experimental observations are yet to be made. We recently proposed that 
thin-film bismuth is a good candidate for the 2D QSH phase [15]. Another candidate 
for the 2D QSH phase is the CdTe/HgTe/CdTe quantum well proposed [16]. The QSH 
phases are, however, yet to be realized experimentally. Distinction between the QSH 
and ordinary insulating (i.e. spin Hall insulator) phases is the absence or presence of 
(topologically protected) boundary states, and is characterized by the Z 2 topological 
number of the bulk states [9j [T71 [18] . 

In the previous paper [19] we considered a phase transition between the QSH and the 
insulating phases in two dimensions. It was found that the phase transitions are classified 
into two cases, corresponding to the presence or absence of the inversion symmetry (I- 
symmetry) in the systems. It was also found that the effective theory describing the 
phase transition consists of two decoupled theories of two-component fermions. 

In this paper we study the phase transition between the two phases in three 
dimensions in the bulk. At first sight it might be similar to the two-dimensional case 
studied in [19]. We find in this paper that it is not. Topological properties such as the 
present problem can be very different for different dimensions. In 3D, the inversion- 
symmetric case is similar to that in the 2D. Nevertheless the inversion-asymmetric case 
is different; when we vary an external parameter which drives the phase transition, there 
appears a gapless phase between the QSH and the insulating phases. An existence of 
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this gapless phase is enforced from a topological origin, and this phase does not exist 
in 2D. These discussions not only deepen our understanding toward the Z 2 topological 
numbers, but also imply some hints for materials search for the 3D QSH phases. 

The paper is organized as follows. In Section 2 we discuss generic phase transitions 
between the two phases in 3D. In Section 3 we discuss the implications of the present 
theory for materials search for QSH phases. Section 4 is devoted for conclusions and 
discussions. We neglect effects of interactions and impurities in the present paper. 

2. Phase transitions between the quantum spin Hall and insulating phases 
in 3D 

2.1. Introduction on the quantum spin Hall phase and Z 2 topological number 

We first review the QSH phase and Z 2 topological number v [HI [9], The simplest example 
of the QSH phase can be realized by a superposition of two quantum Hall systems for the 
up- and down-spins having opposite (effective) magnetic field (see figured]). Suppose 
for the up-spin (down-spin) subsystem the quantum Hall conductance is o~\ y = e 2 /h 
( a xy = ~ e2 /h )• The whole system then has edge states with two spins propagating in 
the opposite direction. The whole system is T-symmetric, and the effective magnetic 
field can be realized by the spin-orbit coupling. 




quantum spin Hall system 



Figure 1. Schematic picture of the quantum spin Hall system as a superposition of 
two quantum Hall systems. 

It is only the simplest special example. In general the QSH allows a spin-orbit- 
coupling term which mixes spins, without breaking T-symmetry. The QSH phase is 
defined as a T-symmetric system which is gapful in the bulk and gapless in the edge. 
Because of the T-symmetry, the edge states form Kramers pairs, consisting of two states 
with opposite spins counter-propagating from each other. The Z 2 topological number 
v is used for distinguishing this QSH phase from the usual insulator phase, namely the 
SH Iphase. The Z 2 topological number can take only two different values for v: even 
or odd, and this number simply means whether the number of Kramers pairs of edge 
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states is even or odd. If v =odd, the system is in the QSH phase, while if v =even the 
system is in the insulating phase. This means that the system with an even number 
of Kramers pairs of edge states is equivalent to a system with no gapless edge state. 
It follows because general perturbations preserving T-symmetry can open a gap in the 
edge states when v =even. In contrast, for the QSH phase {y =odd), the edge states 
remain gapless even in the presence T-symmetric perturbation, including nonmagnetic 
impurities and/or interaction, as long as they are not too strong [T3], [T4"] . 

There are a number of equivalent expressions for the Z 2 topological number 
[TTJ [T71 [18] , and we only explain the ones relevant for the subsequent discussions, both 
in 2D and in 3D. We assume that the spectrum of the Hamiltonian has a gap, within 
which the Fermi energy Ep is located. 

First we explain the 2D case. For I-asymmetric systems, the spectrum is doubly 
degenerate only at the four points k = ki = G/2 {i = 1, 2, 3, 4), and non-degenerate at 
other points. In such systems, the Z 2 topological number v is determined as 

4 



n* 



(i) 



where 



_ y/det[w(fc t )] _ 

6i ~ Pf [«,(**)] " ±L {2) 
Here w(k) is a unitary matrix with elements given by w mn (k) = (u_ k , m \Q\u ktn ) , and 
\uk,n) is the Bloch wavefunction of an n-th band whose eigenenergy lies below Ep. is 
the time-reversal operator, represented as = io~ y K with K being complex conjugation. 
The branch of the square root of the determinant is so chosen that the wavefunctions 
(including their phases) are continuous in the whole Brillouin zone. 

On the other hand, in I-symmetric systems, the formula simplifies drastically; there 
is no need to calculate the phases of the wavefunctions for the whole Brillouin zone, 
which is advantageous for numerical calculation. It is given by 

4 N 

(-If = I]> h= H&rnih), (3) 

i=l m=l 

where £, 2m (ki) (= ±1) is the parity eigenvalue of the Kramers pairs at each of these 
points, and N is the number of Kramers pairs below Ep. 

In 3D, there are four Z 2 topological numbers written as z/ ; {y\v 2 vj) [T2J |Tl] , given 

by 

8 

(-ir = n^> (- i r= n 5 m™ } , (4) 

i=l n k =l;n j7 t k =0,l 

where 5i=( nin2n3 ) (= ±1) is defined for the wavevector ki = \{nibi + n 2 b 2 + n 3 b 3 ) 
{rii = 1,2,3) and bk (k = 1,2,3) are the primitive vectors of the reciprocal lattice. 
These eight wavevectors satisfy ki = —ki (mod G). These topological numbers in 
3D determine the topology of the surface states for arbitrary crystal directions [11]. 
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We note that among the four Z 2 topological numbers in 3D, only v is robust against 
nonmagnetic impurities, while the others {Pf. (k = 1,2,3)) are meaningful only for a 
relatively clean sample [TT] . 

2.2. Phase transitions between the quantum spin Hall and insulating phases 

The problem of our interest in this paper is how the Z 2 topological number changes 
with a change of an external parameter. For the I-symmetric systems it is easier to 
consider; because it is the product of the parity, it can change when the valence band 
and the conduction band with opposite parities touch and exchange their roles. On the 
other hand, in I-asymmetric systems it is not obvious from (j2J) how the Z 2 topological 
number changes. For 2D, this was studied in the previous paper [TS], using the homotopy 
characterization of the Z 2 topological number in [12]. It is defined in the similar way 
as the Chern integer [12], but with some modification. From this definition, it follows 
that the Z 2 topological number in 2Dcan change when the valence and conduction 
bands touch each other at some k = ±fe - (The band touching occur simultaneously at 
k = ±ko because of the T-symmetry.) In 2D, the effective Hamiltonian in the vicinity 
of the phase transition reduces to 

H = E (m, k x , k y ) ± (m - m )a z + (k x - k xQ )a x + (k y - k yQ )cr y (5) 

after unitary and scale transformations, where m is an external parameter which controls 
the phase transition [19]. Equation (jSJ) describes the simplest and general case for the 
band crossing, which occurs at (m, k x , k y ) = (mo, k x o, k y o). 

This story should be modified when the spatial dimension is three, especially for 
the I-asymmetric systems. This is because (JSJ) cannot accomodate the four parameters 
m, k x , k y , and k z , in contrast with the 2D counterpart (JSJ). In the following we answer 
this question, by finding that there should lie a gapless phase between the two gapped 
phases. This gapless phase is a topological phase in the following sense. The phase 
transition is governed by monopoles, namely band-crossing between two non-degenerate 
bands. Such monopoles are topological objects which can appear or dissappear not by 
itself, but by creation/annihilation of a pair of a monopole and an antimonopole. 

We first explain the framework for describing the phase transition in 3D. As in the 
previous paper [19] , for the purpose of describing generic phase transitions between the 
QSH and the insulating phases, we consider a single parameter m which controls the 
phase transition. This parameter m may be considered as externally controllable, and 
by changing this parameter the system undergoes the phase transition. At the phase 
transition the Z 2 topological number must change, which necessitates the closing of the 
gap at some wavevector k. There are various kinds of band crossing when we introduce a 
number of parameters; nevertheless, to study the phase transition, we restrict ourselves 
to "generic" band crossing, and exclude band crossing achieved only by tuning more 
than one parameters. 
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To be specific, we consider a Hamiltonian matrix 

™-(K2)B0- 

where k = (k x , k y , k z ). We assume that the spectrum of the Hamiltonian has no extra 
degeneracies other than those imposed by symmetry. We also assume that the Fermi 
energy Ep lies within a gap of the Hamiltonian. The T-symmetry implies, 

H{k) = a y H T (-k)a y , (7) 

i.e. h-\i(k) = hj\(—k), h^{k) = —h7,(—k), h^(k) = —hJJ—k). The Kramers theorem 
guarantees that the band structure of such T-symmetric spin-1/2 system is symmetric 
with respect to k <-» — k. For the respective cases considered, it suffices to choose 
the dimension of the Hamiltonian matrix to be the number of states involved in band- 
crossing. 

In 3D, as well as in 2D, there is another symmetry which is crucial for the nature of 
the phase transition: the inversion (I-)symmetry. It is because it is the only symmetry 
beside the T-symmetry which transforms between k and — k for all k. We only 
consider these two (T- and I-) symmetries in this paper. Various kinds of band crossings 
found in higher point-group symmetries may be considered as degenerate cases of the 
generic cases considered here. From the Kramers theorem for T-symmetric systems, 
the spectrum is doubly degenerate at the eight points k = fc, = G/2 (i = 1, • • • , 8) in 
I-asymmetric systems, and is so for every k in I-symmetric systems. 

Let us explain the phase transitions in 2D as obtained in the previous paper 
[19]. In I-asymmetric systems the band crossings occur at k = ±fco ^ G/2, between 
non-degenerate bands (figure [2]). Because of the T-symmetry, band crossing occurs 
simultaneously at k — ±fccb & t a single point m = mo- On the other hand, in I- 
symmmetric systems, the band crossing occur at k = ki = G/2 between two doubly- 
degenerate bands. The two bands should have an opposite parity, and their parities are 
exchanged at the band crossing [T9] . In the following we show that the phase transition 
in the I-symmetric systems are similar between 2D and 3D, whereas in the I-asymmetric 
systems they are quite different. 



2.2.1. I-asymmetric systems In 3D, in contrast with the 2D case, band crossing at 
k = ±fc 7^ G/2 cannot lead to phase transion. The reason is the following. The energy 
bands for the I-asymmetric systems are nondegenerate for k 7^ G/2. A crossing of two 
such energy bands has codimension three; namely, by tuning three parameters one can 
make two bands degenerate [201 EI]- To see this, let us consider 2x2 Hamiltonian 
matrix 

where a, b are real functions of k and m, and c is a complex function of k and m. A 
necessary condition for the two eigenvalues to be identical consists of three conditions 
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Figure 2. Phase transition in 2D between the quantum spin Hall (QSH) and insulating 
phases for (a) inversion-asymmetric and (b) inversion-symmetric cases. In the case (b) 
all the states are doubly degenerate. 



a = b, Rec = and Imc = 0, i.e. the codimension is 3 [20]. These three conditions 
determine a curve in the four-dimensional space (m, k x , k y , k z ). Thus for generic m there 
will be in general a point (or points) k where the eigenvalues are degenerate. When 
m is changed continuously the k point moves in the k space, and the system remains 
gapless. It will be revealed later how the system can open a gap and run into either the 
QSH or the insulating phases. 

On the other hand, for k = G/2, band crossing cannot occur in general. At the 
points k = G/2, the spectrum is doubly degenerate, and the codimension is five [22| [23] . 
It can be explicitly seen as follows. As the number of states involved is four, we consider 
4x4 Hamiltonian matrix with the constraint ([7]). It leads to a result 

5 

H(k = k i ) = E + ^2a i T i (9) 

i=l 

where Oj's and Eq are real, and T% = 1 ® r x , r 2 = cr z ® t v , T 3 = 1 <g> r z , r 4 = a y <g> r y , 

and r 5 = a x ® r y . Its eigenenergies are given by E ± v/X)i=i a 1- The two (doubly- 
degenerate) bands will touch when a, = for i = 1, ■ • • , 5, which are not satisfied by 
tuning only one parameter m. (Note that the wavenumber k is fixed here and cannot 
be changed.) Thus it is impossible to control the bands to touch at k = G/2 by tuning 
a single parameter m. 



2.2.2. I -symmetric systems In I-symmetric systems, the energies are doubly degenerate 
for every k by the Kramers theorem. The phase transition occurs when the gap between 
the two doubly-degenerate bands closes at some k. Because there are four states 
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involved, we consider the 4x4 Hamiltonian matrix H(k). We impose the I-symmetry 

as 

H(-k) = PH(k)p-\ u(-k) = Pu(k), (10) 

where P is a unitary matrix independent of k, and u(k) is the periodic part of the 
Bloch wavefunction: v 9 fc( r ) = u(k)e tkr . After a judicious unitary transformation, all 
cases reduce to 

P = ^ A p \ P] =Pl = diag(r/ a , V b) (11) 

without losing generality r) a and r\ h represent the parity eigenvalues of the atomic 
orbitals involved. 

The band crossings are different for i] a = rjb and r\ a = —r\b in 3D, as is similar to 
2D [19]. When r] a = rjb = ±1, the generic Hamiltonian becomes 

5 

H(k)=E (k) + J2^(k)Ti (12) 

i=l 

where a^'s and Eq are real even functions of k. On the other hand, when rj a = —r\b = ±1, 
the Hamiltonian reads, 

4 

ff(fc) = Eo(fe) + a 5 {k)T' 5 + 6 (<) (fc)r; (13) 

i=i 

where E (k) and a 5 (fc) are even functions of fc, fo^(fe) are odd functions of fc. The 
matrices r' x = <r 2 <8> t x , r' 2 = 1 <g> r y , T' 3 = a x ® r x , T' 4 = a y ® r x , and 1^ = 1 <8> r 2 
form the Clifford algebra. Therefore, for generic point k = k { ^ G/2, In both cases, 
r] a = rjf, and r\ a = —77^, the codimension is five, which exceeds the number of tunable 
parameters (m, k x , k y , k z ). Therefore, band crossing does not occur at a generic point 
k with k^ki = G/2, 

On the other hand, at the high-symmetry points k = ki = G/2. The number of 
parameters to achieve degeneracy is five for r\ a = r)b (in ([121) ) while it is one for r\ a = —r/b 
(in (fTBl ). Because the wavenumber is fixed k = ki = G/2, there is only one changeable 
parameter m. Thus only when rj a = —T)b, the two-doubly degenerate bands touch at 
k = ki = G/2. This situation is similar to the case in 2D [T9] . 

Thus we have seen that in I-symmetric 3D systems, the phase transition can occur 
at a single value of the parameter m = m . At one side (e.g. m < m ), the system is in 
the QSH while the other side (e.g. m > mo) is in the insulating phase. Meanwhile, the 
phase transition at a single value of m is in general impossible in I-asymmetric cases. 
Here we encounter a question. How is the phase transition in the I-symmetric 3D system 
modified when some perturbation breaks I-symmetry. Because the two sides [m m$ 
and m 3> m ) belong to the different phases, there should be a phase transition in 
between. 

The answer is the following. Instead of a phase transition occuring at the single 
value of the parameter m, there appears a finite region of m where the system remains 
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gapless. As was discussed previously, the band crossing in the I-asymmetric system 
cannot occur at a single value of m. The gapless points in the four-dimensional m-k 
space forms a one-dimensional manifold (i.e. a curve). Thus, the only way to close the 
gap by changing m is to make a pair of gapless points, as shown in figure [3j 



(a) k A 




1 > 

m<m m=m m>m it 



Figure 3. Location of the the gapless points by changing the external parameter m 
in (a) I-asymmetric systems and (b) I-symmetric systems. In (a) the green and the 
red denotes trajectories of the monopole and antimonopole, respectively. 



To see the behavior of such gapless points, we note that each gapless point carries a 
topological number. Such gapless point, sometimes called a diabolical point, is regarded 
as a monopole in the k space [23 EE]- Indeed, the Bloch wavefunctions with 
nondegenerate spectrum can be associated with a U(l) gauge structure in the k space; 

A n (k) = - i(kn\V k \kn} } (14) 
B n (k) = V fc x A n (k), (15) 

p n {k) =^-V k -B n (k) (16) 

Z7T 

The quantity p(k) is called a monopole density. It vanishes when the n-th band is not 
degenerate with other bands. At the band crossings between the n-th band and other 
bands, p{k) has a (^-function singularity; p n {k) = J2i Qin^(k — ki n ) where q\ n is an integer 
called a monopole charge. For example, for the band crossing at k = ko with linear 
dispersion (Weyl fermion) 

3 

H = E (k)+J2Mk)^ (17) 

i=l 

where fi(k = k ) = 0, the monopole charge for the lower band is sgn(det(|^)jj)|fc=fc 
(=±1). 
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In the present system, a pair of a monopole (charge q = 1) and an antimonopole 
(charge q = — 1) is created at m = mi, k = ki = G/2 when m is increased, and 
the system becomes gapless. When m is increased further, the monopole (k = k + ) 
and the antimonopole (k = fc_) moves in the k space, while the T-symmetry imposes 
that k = —k + . This system can open a gap again only when the monopole and 
antimonopole annihilate together. The annihilation can occur only at the points with 
k = ki = G/2, again by the T-symmetry (figure [3]). Thus the overall feature of the 
phase transition is schematically expressed as in figure HI 



2.2.3. Summary for the phase transition in 3D To summarize, the overall feature 
of the band crossing is schematically shown in figure HI The overall phase diagram 
is schematically shown in figure El in a plane of the control parameter m and another 
parameter 5 representing an inversion-symmetry breaking. As we have seen, when the I- 
symmetry is broken, the topological gapless phase appears between the two phases which 
are gapful in the bulk. The difference of the Z 2 topological numbers v between the two 
sides of the phase transition can also be calculated as in the 2D case [19] . When the band 
crossing occurs at ki = ^(^l&i + n 2 b 2 + n 3 b 3 ), the factor 6~i=( ni n 2 n 3 ) in (Tj0) changes sign, 
and some of the four Z 2 topological numbers, Uq] (^iz/ 2 z/ 3 ) in (TJJ, change accordingly. 
This applies to the I-symmetric systems. The I-asymmetric cases are similarly treated 
because it can be associated with the I-symmetric cases with perturbation. 
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Figure 4. Phase transition in 3D between the quantum spin Hall (QSH) and insulating 
phases for (a) inversion-asymmetric and (b) inversion-symmetric cases. In the case (b) 
all the states are doubly degenerate. 
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inversion 
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(5 =0) 
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Figure 5. Phase diagram for the quantum spin Hall (QSH) and ordinary insulating 
(I) phases for (a) 2D and (b) in 3D. m is a control parameter which drives the phase 
transition, and 5 represents a parameter which describes the breaking of inversion 
symmetry. S — is the case with inversion symmetry. 



3. Towards materials search for quantum spin Hall systems 

3.1. Bismuth thin film and 2D quantum spin Hall system 

In [15], the bilayer bismuth is studied as a candidate for the 2D QSH phase. From the 
2D bilayer tight-binding model, truncated from the 3D tight-binding model [27], the 
bilayer bismuth is proposed to be the 2D QSH system [15]. Bilayer antimony is studied 
in a similar way, and it is the ordinary insulator. It is predicted from the calculation 
of Z 2 topological number and from a band structure calculation for the geometry with 
edges (i.e. the strip geometry). The Z 2 topological number is calculated in [15] by the 
Pfaffian of the matrix for the time-reversal operator proposed in [8]. 

The calculation of the Pfaffian involves fixing of phases of the wavefunction as an 
analytic function of the wavenumber k, which is numerically a challenging problem, even 
for a simplified model presented in [15J. It can be tackled by discretizing the k space and 
counting the vortex of the Pfaffian matrix [28l [29] . Instead, for I-symmetric systems, 
the method of calculating parity eigenvalues ([3]) proposed in [18] is much easier. For the 
bilayer bismuth, we checked that this method leads us also to the same conclusion that 
the Z 2 topological number is odd and nontrivial, and it is in the QSH phase. 

For the system to be the QSH phase, it should have a gap in the bulk. The 3D 
bulk bismuth is semimetallic, and have a small band overlap between the conduction 
and the valence band, while the direct gap is finite for all wavenumbers. By making it 
into thin film, the perpendicular motion is quantized and tends to open a band gap. In 
earlier theoretical estimate [30], [31] and experiments [32], [33] (see also [33]) shows that 
in thin-film bismuth thinner than ~ 30nm becomes an insulator. However, in recent 
experiments by Hirahara et al. [35] the angle-resolved photoemission spectra (ARPES) 
the ultrathin films of 10 bilayers are measured. In their experimental data, as opposed 
to the earlier theories and experiments, the system remains semimetallic. It casts us 
a question that the critical thickness might be much smaller than the earlier estimate 
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[3DIEDE21E3]. 

3.2. Bismuth and 3D quantum spin Hall system 

In [18] it was suggested that the 3D Bi has v =even, while the 3D Sb has v =odd. 
This looks opposite to the case of the 2D. To resolve this, numerical analysis was 
performed in [29] by artificially changing the interlayer hopping multiplied by a factor 
/ (0 < / < 1). It was found that for Bi, the regions < / < 0.223, 0.223 < / < 0.993, 
and 0.993 < f < 1 are the phases 0;(111), 1;(111), and 0;(000), respectively, whereas 
for Sb the regions < / < 0.54 and 0.54 < / < 1 are the phases 0;(000) and 1;(111). 
This calculation was done by the calculation of the Pfafnan matrix, while this result was 
checked by the parity analysis proposed in [18]. The phase transitions by changing / 
are described by our theory developed in the previous section. The band crossing which 
accompanies the phase transition occurs for Bi at (0, 0, 0) for / = 0.223, and at (tt, 0, 0), 
(0,7r,0), and (0,0, n) for / = 0.993. For Sb, the band crossing occurs at (7r,7r,7r) for 
/ = 0.54. These systems are I-symmetric. According to our theory, the band crossing 
occurs only at the single value of the control parameter /, rather than having a gapless 
phase in between. 

In reality, both Bi and Sb are semimetals, not insulators. The Z2 topological 
numbers are defined in the 3D, by assuming that the band overlap is lifted by some 
perturbation, thereby the bands below the gap are regarded as the "valence band" 
which enters in the definition of the Z2 topological numbers. This is possible because 
there is a direct gap in every k. In this sense, although the 3D Z 2 topological number 
vq defined as such is odd (nontrivial) in Sb, it is not the QSH phase, because there is 
no gap. If one can open a gap in the 3D Sb by some external perturbation, it becomes 
the QSH phase. 

Nevertheless, because the Z2 topological numbers can be defined in Bi and Sb in 
the above sense, they manifest themselves in the spectrum of surface states. The 3D 
bulk Bi and Sb has the topological numbers 0;(000) and 1;(111), from which expected 
topology of the Fermi surface of the surface states can be easily sorted out for various 
directions of crystal surface [18] . The results can be compared with the experiments on 
the angle-resolved photoemission spectroscopy (ARPES) for Bi (for example, [36]) and 
for Sb [37]. To interpret these experiments to see whether it matches the prediction 
from the Z2 topological number, we need to separate the Fermi surface of bulk states 
and that of surface states, which is not trivial experimentally. 

To find a clear experimental manifestation for the nontrivial Z2 topological number, 
we need anyway open a gap in the bulk. One example is Bii^Shj, (0.07 < x < 0.22), 
and ARPES experiments on this doping region is called for. To our knowledge, [38] 
is the only experimental report in this doping region. Nonetheless, this experiment is 
limited only in the vicinity of the k = 0, and not sufficient to examine its topological 
phase from the ARPES data. A data on ARPES in Bii.^Sb^ (0.07 < x < 0.22) covering 
the whole surface Brillouin zone would be a clear evidence for the QSH phase. Thus, 
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there is no clear experimental evidence for the QSH phase as yet, whereas it looks well 
within our reach, and a good candidate is strongly called for. 

3.3. Criterion for searching quantum spin Hall systems 

Apart from the time-reversal symmetry, there are only two conditions for the QSH 
phases. 

(i) the bulk is a band insulator 

(ii) the Z 2 topological number is odd 

The first condition is clear, while the second requires calculation. At this stage we need 
some strategy to search among the vast number of nonmagnetic insulators. 

To find out the strategy, let us begin with a system without spin-orbit coupling, and 
switch on the spin-orbit coupling gradually. Insulators without the spin-orbit coupling 
have a trivial (i.e. even) Z 2 topological number. To reach the QSH phase, the gap 
should once closes in switching on the spin-orbit coupling, thereby the system should 
undergo a phase transition. This phase transition should be described within our theory 
developed in the previous section. 

Therefore, the QSH phase near the phase transition may have the direct gap at 
k = ki = G/2, as a trace of the phase transition. When this phase transition occurs 
similtaneously at some points in the Brillouin zone, the changes of the Z 2 topological 
number add together. In this sense, when the material has an even number of equivalent 
points for the direct gaps which close at the phase transition, the Z 2 topological number 
vq does not change at the transition, and the system remains an ordinary insulator. The 
example is the 4 equivalent L points in PbTe |18| . and PbTe is indeed in the ordinary 
insulator phase. 

Therefore it is desirable to have direct gaps at an odd number of points in the 
Brillouin zone, such as the L points in Bi 1 „ x .Sb ;E [18], or the T point in generic crystals. 
We note that this is not a necessary condition, but can be a reasonable guideline for 
searching candidate materials among nonmagnetic insulators. 

It is also necessary to see whether the gap in the material considered is "after" 
or "before" the phase transition, when one turns on the spin-orbit coupling gradually. 
In other words, for the QSH phase, the gap should be originated from the spin-orbit 
coupling. This statement is somewhat vague; to make this more transparent, the author 
suggested that the susceptibility can be a measure to see whether the gap is of the 
spin-orbit nature [15]. For example, bismuth are strongly diamagnetic, because of the 
inter-band matrix elements between the conduction and the valence bands due to the 
spin-orbit couping. This class of materials, when gapped, are good candidates for the 
QSH phases. The criteria discussed so far will be useful for finding good candidates for 
the QSH phase. 
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4. Conclusions and Discussions 

In the present paper we studied the phase transition between the QSH and the insulating 
phases in 3D. In contrast to the 2D systems, in the 3D inversion-asymmetric cases there 
is a gapless phase between the two phases. This gapless phase originates from the 
topological nature of the monopoles (band-crossing points) in 3D, which are responsible 
for the phase transition. Furthermore, the gap closing occurs only at the high-symmetry 
points k = G/2, which is also in contrast with 2D cases. 

So far we have given topological argument in terms of the Bloch wavef unctions. 
One may suspect that the scenario may become invalid in the presence of interaction 
or disorder, thereby the topological order may be obscured. We give here qualitative 
argument that our scenario is robust against small perturbations. Our argument is based 
on the fact that the Z 2 topological number (for gapped systems) can be defined even 
in the presence of interaction and disorder [18] , by use of an analogue of the Laughlin's 
gedanken experiment [39]. In disordered systems, the Bloch wavenumber k x becomes 
ill-defined. We then think of folding the system to a "ring" , periodic in the x direction, 
and threading a flux $ x into its hole. The flux $ x plays the role of k x . A similar 
procedure is taken for k y and k z . The Z 2 topological number is defined by using $j 
(i = x, y, z) instead of fcj. Therefore, as long as interaction and disorder is relatively 
weak, the bulk remains gapped and the Z 2 topological number remains well-defined and 
idential with that in the clean, noninteracting systems. As is related with this stability 
of the topological order, the gapless surface state in the 3D QSH phase remain gapless 
and show antilocalization behaviour in the presence of impurities. It is in the symplectic 
universality class, and is similar to the honeycomb lattice without intervalley scattering 
|40j . For this reason, the 3D QSH phase is robust against impurities. 

Based on these observations, we can now see that the gapless topological phase 
(mi < m < m 2 in Figure H^a)) will survive in the presence of disorder and interaction. 
Since the two sides of the topological gapless phase are the gapped phases (m < m\ 
and m > m 2 in Figure Ufa)) with different Z 2 topological numbers, there should lie 
a gapless phase in between, even in the presence of interaction and disorder. (If 
the topological gapless phase becomes gapped, it can no longer accompany a phase 
transition.) Quantitative argument to see how this gapless phase is robust is involved 
and is beyond the scope of the present paper, 

A following remark is in order. A calculation for the Z 2 topological number based on 
a simplified model requires some care, because the bands well below the Fermi energy 
might contribute to the Z 2 topological nubmer. We present one example. In [7] we 
constructed a 4-band tight-binding model for HgTe. It is a zero-gap system, but by 
opening a gap with uniaxial pressure, it becomes a band insulator. The question here 
is whether it is the simple insulator or the topological insulator (i.e. QSH). In fact, 
from the parities of the bands below the Fermi energy Ep from the four-band model in 
[7], the Z 2 topological number v turns out to be even. It disagrees with the result in 
|18j . The reason for the disagreement is the following. This model only has four bands 
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near the Fermi energy Ep, and the other bands well below Ep are discarded. In HgTe, 
one doubly-degenerate band just below the Fermi energy has a trivial Z 2 topological 
number, while the other discarded bands deep below the Fermi energy has a nontrivial 
Z 2 topological number; this leads to the seemingly contradicting results. Thus in the 
calculation of the Z 2 topological number one should be careful because it involves all 
the bands below the Fermi energy Ep. 

The lesson we can learn from the examples of bismuth [15J and other materials [18] 
is that the materials with odd Z 2 topological numbers are not rare. In nature there 
may well exist systems with odd Z 2 topological number. The crucial difference between 
the two kinds of topological insulators, namely the quantum Hall (QH) and the QSH 
system, is that the QH systems require magnetic field as strong as several teslas, whereas 
the QSH systems does not require external fields to achieve topological phases. This is a 
crucial difference. To realize the QSH phase, the effective magnetic field produced from 
the spin-orbit coupling should be of some teslas, as expected from the analogy with the 
QH system. We see from the examples of bismuth [15] and other materials [IE] that 
the spin-orbit couping is strong enough in some materials. Thus it is natural to expect 
some materials in nature to be in the QSH phase, which would be of great interest both 
theoretically and experimentally. 
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